fix(bedrock-converse): parse streaming tool_kwargs from string to dict#21580
fix(bedrock-converse): parse streaming tool_kwargs from string to dict#21580ayushh0110 wants to merge 1 commit intorun-llama:mainfrom
Conversation
Bedrock ConverseStream delivers tool use input as string chunks. The streaming code correctly concatenates these into a complete JSON string, but passes the raw string directly to ToolCallBlock.tool_kwargs instead of parsing it into a dict. This breaks cross-provider workflows where chat history from a Bedrock streaming session is replayed through another provider (e.g. Google GenAI) that expects tool_kwargs to be a dict. Add _parse_tool_input() helper that safely parses JSON strings to dicts and apply it at all 6 ToolCallBlock construction sites in stream_chat and astream_chat. Fixes run-llama#21579
dosubot
Bot
added
the
size:S
|
This looks like a solid fix! The approach is clean, correct, and well-tested. A few observations: What works well:
On the approach difference from the Dosu suggestion: You noted you went with wrapping at each construction site instead of a single One minor note: intermediate yields during streaming will have incomplete JSON strings that Overall this is a clean, minimal fix that directly addresses the cross-provider incompatibility reported in #21579 [3]. Nice work! 🎉 To reply, just mention @dosu. Docs are dead. Just use Dosu.
|
1 participant
Description
Fixes #21579
Bedrock
ConverseStreamdelivers tool use input as partial JSON string chunks viaToolUseBlockDelta.input. The streaming code (stream_chat/astream_chat) correctly concatenates these chunks into a complete JSON string, but passes the raw string directly toToolCallBlock.tool_kwargsinstead of parsing it into a dict.This breaks cross-provider workflows where chat history from a Bedrock streaming session is replayed through another provider (e.g., Google GenAI), which expects
tool_kwargsto be adict:The non-streaming path works fine because boto3 automatically deserializes
ToolUseBlock.inputto a Python dict.Changes
_parse_tool_input()helper function that safely parses JSON strings to dicts (with fallback to{}for incomplete/malformed JSON during intermediate streaming yields)ToolCallBlockconstruction sites instream_chat(3 sites) andastream_chat(3 sites)test_stream_chat_tool_kwargs_parsed_as_dictandtest_astream_chat_tool_kwargs_parsed_as_dict) that mock the full Bedrock streaming tool use event sequence and asserttool_kwargsis adictWhy this approach
The @dosu analysis suggested parsing at the
contentBlockStopevent. However, the current code has nocontentBlockStophandler, and intermediatecontentBlockDeltayields would still emit raw strings before that event fires.Instead, wrapping at each construction site:
get_tool_calls_from_response()(which already doesisinstance(tool_call.tool_kwargs, str)→parse_partial_json)Test results
All existing tests pass with zero regressions.